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ABSTRACT 

If  in  a  control  system  the  plant  is  described  by  ^  =  A;^  4-  Bu,  1  |  £  1,  and  if 

t  r 


during  a  control  action  it  is  desired  to  minimize  the  functional  F 


DidT, 


j=l 


then  it  is  shown  that  each  control  signal  u^(t)  must  be  piecewise  constant,  with  the 
values  4- 1,  0,  or  -■  1,  and  that  each  control  signal  is  a  nonlinear  function  of  the  adjoint 
system.  If  each  control  signal  is  proportional  to  rate-of-flow  of  fuel  (or  mass),  the 
results  obtained  lead  to  a  "minimum -fuel"  control  system.  Plants  with  single  control 
inputs  arc  also  analyzed. 

The  theory  is  applied  to  the  analysis  and  design  of  the  minimum-fuel  controller 
for  two  specific  second-order  plants.  The  first  plant  has  the  transfer  function  1/s^ 
and  the  second  l/(s  +  l)(s  +  2).  The  optimum  feedback  control  function  is  obtained  as  a 
function  of  the  plant  state  variables. 
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INTRODUCTION 


In  the  past  five  years  the  theory  of  control  systems  has  been  influenced  by  a 
host  of  elegant  mathematical  techniques,  mostly  based  on  the  Calculus  of  Variations. 
Perhaps  the  most  celebrated  theoretical  result  is  the  "Maximum  Principle"  originated 
by  Pontriagin  and  his  colleagues,  which  is  related  to  the  theory  of  Dynamic  Programming. 

The  best  known  optimum  control  system  is  the  minimum  time  one,  or  as  it  is 
commonly  called,  the  "bang-bang"  control  system.  During  the  last  year,  however,  the 
attention  has  shifted  to  other  optimal  control  systems,  such  as  the  minimum  energy  one. 

In  this  report  wc  examine  the  so-called  "minimum -fuel"  optimum  control 
system.  We  are  concerned  with  systems  for  which  the  control  signal  is  proportional 
to  fuel,  such  as  fuel  exhaust  in  a  missile  or  gas  exhaust  from  a  gas  jet  in  an  attitude 
control  system.  The  total  fuel  consumed  during,  a  control  action  is  proportional  to 
the  time  integral  of  the  absolute  value  of  each  control  signal.  If  we  are  given  a  linear 
system  which  is  to  execute  a  given  task,  then  we  are  interested  in  determining  the 
control  signal  which  requires  the  least  amount  of  fuel  to  accomplish  the  given  task. 

We  also  assume  that  each  control  signal  is  bounded  in  magnitude  by  unity. 

In  Section  1  of  this  report  we  define  the  problem  mathematically,  and  we 
state  the  theoretical  results  which  will  be  employed.  In  (1.  5)  we  derive  the  optimal 
control  law  for  minimum  fuel  operation  which  states  that  each  control  signal  is 


piecewise  constant  and  is 


if  unity  is  the  bound  on  each  control,  and 


that  the  magnitude  and  polarity  of  each  control  signal  is  a  nonlinear  function  of  the 
output  of  the  adjoint  system. 
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In  the  second  Section  we  examine  the  minimum  fuel  control  of  a  plant  described 
th 

by  an  n  -  order  linear  differential  equation  with  constants  coefficients.  We  use  the 
similarity  transformation  in  order  to  define  a  convenient  set  of  state  variables . 

In  the  third  Section  we  use  the  theory  to  design  a  minimum  fuel  controller  for  a 
double  integral  plant.  The  system  is  analyzed  using  the  trajectories  in  the  phase  plane 
and  utilizing  the  information  obtained  from  the  adjoint  system.  The  controller  is  shown 
in  Fig.  8. 

In  the  fourth  Section  we  examine  In  great  detail  the  optimum  minimum  fuel  control 
of  a  second -order  plant  with  tw'>  real,  negative,  and  distinct  poles.  Through  the  use  of 
phase-plane  trajectories,  and  using  the  concept  of  equal  fuel  lines,  we  arrive  at  a 
separation  of  the  phase  plane  into  three  regions  of  operation.  For  each  region  of 
operation  there  exists  an  optimal  value  of  the  control  signal. 

This  study  is  not  exhaustive,  A  general  method  for  computing  the  optimal 
control  for  minimum  fuel  operation  of  any  system  is  not  given.  However,  the  examples 
presented  may  be  used  to  determine  the  optimal  control  of  similar  types  of  systems, 
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1.  MATHEMATICAL  THEORY 


1. 1  introduction  . 

In  this  Section  the  problem  of  minimum-fuel  control  is  stated  in  precise  terms. 
The  terminology  is  established.  The  available  theory  of  optimal  control  systems  is 
briefly  reviewed  and  is  used  in  order  to  derive  the  optimal  control  law  for  minimum  fuel 
systems . 

1. 2  Statement  of  the  Problem 

We  are  given  a  dynamical  system  described  by  the  matrix  differential  equation 

i  =  Ajr  +  Bu(t).  1-1 

We  shall  refer  to  the  system  described  by  Eq.  ^  -1  as  the  (controlled)  plant.  The 
plant  is  assumed  to  be  linear,  time-invariant,  and  stable. 

InEq.  1-1: 

I  =  {y^*  ^2 . n-column  vector  called  the  state  of  the  plant. 

A  =  (a^j)  is  an  nxn  real  constant  matrix. 

HW  =  r-column  vector  called  the  control. 

B  =  (b^j)  is  an  rxn  real  constant  matrix, 
n  is  the  order  of  the  plant  in  Eq.  1-1. 

We  assume  that  each  element  of  the  control  vector  u(t)  is  bounded  in 
magnitude  by  unity,  i.  e. , 


(u.(t)(<l  ;  j=l,2 . r  1-2 

The  assumption  that  unity  is  the  bound  of  each  element  of  the  control  function  is  for  the 
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1.3  Discussion  of  the  Problem  . 

There  are  many  physical  systems  which  can  be  mathematically  represented 
in  the  form  of  Eq.  1-1.  Time-varying  and  nonlinear  systems  are  excluded.  It  is 
also  assumed  that  the  systems  are  noise-free. 

The  control  signals  Uj(t)  act  as  the  inputs  to  the  controlled  plants.  There 
are  many  cases  for  which  each  control  signal  represents  the  reaction  provided  by  the 
exhaust  of  mass  as  discussed  in  the  Introduction.  Then  the  absolute  value  of  each 
control  signal  physically  represents  rate-of-flow  of  mass  or  fuel.  Hence,  the 
functional  in  Eq.  1-3  represents  the  total  fuel  consumption  necessary  for  the  control 
of  the  plant.  Minimization  of  the  fuel  by  the  appropriate  choice  of  the  controls 
Uj(t)^  leads  to  a  minimum-fuel  control  system.  Throughout  this  report  optimal  control 
system  will  mean  minimum-fuel  control  system. 

The  assumption  on  the  bound  of  the  control  signals,  as  provided  by  Eq.  1-2,  is 
due  to  the  saturation  phenomena  of  power-amplifying  elements  in  electrical  systems, 
or  due  to  other  physical  considerations.  For  example,  in  an  attitude  control  system 
there  exists  a  maximum  torque  which  the  escaping  gas  can  provide  due  to  design 
considerations. 

The  plants  examined  are  assumed  to  be  controllable.  For  a  precise 
mathematical  definition  of  controllability  the  reader  is  referred  to  Refs.  1  and  2. 

Loosely  speaking,  controllability  implies  that  the  objectives  of  the  control  system,  i.  e. , 
the  transfer  of  the  state  from  to  must  be  consistent  with  its  capabilities,  which 
are  determined  from  the  mature  of  the  matrix  A  and  the  bound  imposed  on  the  control 
functions . 
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If  the  system  Is  controllable,  then  there  are,  usually,  many  admissible 
controls  which  will  force  the  plant  from  an  initial  state  to  a  desired  final  state. 

Each  of  the  controls  will  result  in  a  certain  amount  of  consumed  fuel.  The 
optimization  procedure  will  lead  to  the  determination  of  the  control  (s)  which  will 
require  the  least  amount  of  fuel. 

1 . 4  Theory  of  Optimal  Control 

In  this  Section  we  will  state  briefly  the  latest  results  in  the  mathematical 
theory  of  control.  For  details  the  reader  is  referred  to  Refs.  3,  4,  5,  and  6.  The 
terminology  of  Ref.  3  is  used  in  this  section. 

We  arc  given  a  system  described  by  the  differential  equation 

X  =  1  (X>  H(t).  t )  1-6 

where  y  is  the  state  vector  and  u  is  the  control  vector.  Let  denote  the  initial 

state  at  t  =  t  .  Let  the  solution  of  Eq.  1-6  be 
0 

Assume  that  the  control  u(t)  is  an  element  of  a  closed  convex  set  fi,  i.e.,  u(t)  e  Si. 
Let  S  be  a  surface  in  the  state  space.  We  are  given  the  scalar  functional 

V(XQ.to.S  ;  u)  =  \  L(0,u,t)dt  1-8 

^0 

where  tj^  is  the  time  for  which  0(t)  t  S  for  the  first  time. 

It  is  desired  to  find  the  optimal  control  u° (t),  u°(t)  e  n,  such  that  the 
functional  of  Rq.  1-8  is  minimum  with  respect  to  all  other  admissible  controls. 
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The  following  steps  must  be  followed  in  order  to  determine  a  set  of  necessary, 
but  not  sufficient,  conditions  on  the  optimal  control  u°{t). 

Step  h  Form  a  scalar  function  H,  called  the  Hamiltonian, 

H^.£,u,t)  =  LC^.u.t)  +  <  jr,p>,  1-9 


where  £  is  a  vector  called  the  costate,  and  <  ^,  p  > 
is  the  scalar  product  of  the  vectors  ^  and  £  . 

Step  2:  Find  the  absolute  minimum  of  the  Hamiltonian  H  with 
respect  to  all  admissible  controls.  Let 

H’*(j?,£,t)  =  min  H(}r,£,u,t)  1-10 

u  e 

Let  u^  be  the  control  which  minimizes  H.  Then 

H*  =ii*(jr.£,t)  1-11 

Hence, 

H*(x.E.t)  =  H(x>p.u*,t)  <  H(^,p,u,t)  1-12 

for  all  u  ^  u*  . 
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8H 

8e 

8H 


or,  more  succinctly, 


i  = 


Step  4:  Then,  a  necessary  condition  on  the  optimal  control  u°,  which  minimizes 
the  functional  of  Eq.  1-8,  is  that  it  also  minimizes  the  Hamiltonian  H 
of  Eq.  1-9.  Hence, 

u*(jr,£,t)  =  u°(t)  1-17 

represents  a  necessary  condition. 

In  Refs.  3  and  6  sufficiency  conditions  are  also  given  in  the  form  of  a 
Hamllton-Jacobi  partial  differential  equation  with  split  boundary  conditions. 

In  the  next  Section  It  will  become  evident  that  the  theory  above  may  be  used  in 
order  to  determine  the  form  of  the  optimal  control  function  but  not  its  exact  time 
dependence  or  its  dependence  on  the  actual  state  of  the  system,  i.e. ,  the  optimum 
feedback  control  problem. 

1.  5  The  Optimal  Minimum-Fuel  Control  Function 

In  this  Section  we  utilize  the  theoretical  results  of  Section  1-4  in  order  to 
determine  the  form  of  the  optimal  control  function  for  minimum-fuel  operation.  For 
the  minimum-fuel  problem  the  functional  F  of  Eq.  1-3  is  a  special  form  of  the 
functional  V  of  Eq.  1-8.  It  follows  that  for  the  minimum -fuel  problem 


We  form  the  Hamiltonian  H 

r 

H  =  ^  |Uj(t)  1  +  <  2  .P  >  •  1-19 

j=l 

Substituting  Eq.  1-1  Into  1-19  we  obtain 

r 

H  =  ^  ^  ^  4Z’£  >  +  <  Bu,p  >  .  1-20 

j=l 

The  canonical  equations  1-15  and  1-16  yield 

i‘  -  4z  +  5"  1-21 

E=--f-*-4'2  1-22 

where  A'  is  the  transpose  of  matrix  A.  Equation  1-21  is  identical  fo  Eq  .  1-1 , 
which  describes  the  behavior  of  the  state  of  the  plant.  Equation  1-22  is  an 
ordinary  homogeneous  matrix  differential  equation,  which  has  the  solution 

-A'r  A  -A't 

£(t)  =  e  ^  ^£(0)  ^  e  ^ .  1-23 

Equation  1-23  defines  the  time  response  of  the  costate  £.  We  shall  refer  to  the 
system  described  by  Eq.  1-22  as  the  adjoint  system.  Note  that  the  costate  £(t) 
is  independent  of  the  control  u(t). 

Now  we  must  determine  the  control  which  absolutely  minimizes  the 
Hamiltonian  H  of  Eq.  1-20.  First,  note  that 
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1-25 
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c^(t)  =  1  Uj(t)|+  Uj(t)q^(t)  =  1  u^(t)  1  +  sgii  {  qj(t)}  Uj(t)  1  qj(t)  1 


1»30 


Let 


Case  1 


If 


8gn{qj(t)}  =+l. 


(1)  If 


u^(t)  >  0,  then 

Cj(t)  =  1  Uj(t)  I  [  1  +  I  q^(t)  h  , 

which  implies 

Cj(t)  =  0  ,  hence 

min  c,(t)  =  0  is  obtained  by 
Uj  j 


Uj(t)  =  0. 


(ii)  If  u^(t)  <  0,  then 

Cj(t)  =  luj(t)|  [  1  -  lqj(t)  1  1 . 

(a)  If  1  1  -  I  q.(t)  1  1  ^  0,  then 


c^(t)  2=  0,  hence 

min  c,(t)  =  0,  obtained  by 

Uj  j 

U,<t)  =  0. 


(b)  H  ll-lq/'i!  <  0,  then 
f'  (t)  <•  n.  hence 

-.V- 

min  c,(t)  =  [  1  -  1  q.(t)  1  1  is  obtained  by 
Uj  j  J 

lu^(t)  I  =  1. 
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Case  2:  If  sgn  {qj(t)}  *  -  1. 

(i)  If  Uj(t)  >  0,  then 

c^(t)  =  |u^(t)I  tl-lqj(t)h. 

(a)  If  [1-1  qj(t)  h  5:  0,  then 
Cj(t)  >  0,  hence 
nhn  Cj(t)  =  0  is  obtained  by 

uj(t)  =  0 


(li) 


(b)  If  1 1  -  I  q^Ct)  h  <  0.  then 
Cj(t)  <  0,  hence 
min  Cj(t)  =  ( 1  -  I  qj(t)  h 

|uj(t)  I  =  1. 

If  Uj(t)  s  0,  then 

Cj(t)  =  !uj(t)  I  [  1  +  |qj(t)  1  1  , 

which  Implies 

Cj(t)  >  0  ,  hence 

m^n  =  0  is  obtained  by 

u  (t)  =  0. 

J  • 


is  obtained 


by 


From  the  above  results  one  may  state  the  following  Theorem. 

Theorem  b  For  miiiiiiium-fucl  control  of  the  plaiit  deacribed  by  Eq.  1-1,  and  for 
control  signals  restricted  by  Eq.  1-2,  the  optimal  control  functions  are  determined 
by 
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U.(t)  =  0 

Uj(t)  =  -  sgn{qj(t)  } 
for  j  =1,  2, . . .  ,r 


sgn{l  -  lq.(t)  1  }  =  +  l  1 
sgn{l  -  lq.(t)  1  }  =  -  1  S 


and  q^(t)  defined  by  Eq.  1-25. 


The  above  theorem  states  that  the  optimal  control  functions  for  minimum-fuel 
control  must  have  the  values  +1,  0,  or  -  1,  as  indicated  by  Eq.  1-31.  Hence,  the 
optimal  control  is  piecewise  constant.  The  magnitude  and  the  polarity  of  each  control 
signal  depend  on  the  costate  p,  throu^  Eq.  1-25,  and  hence,  on  the  output  of  the 
adjoint  system. 

1. 6  Discussion  of  the  Results  . 

The  results  obtained  in  Section  1. 5  are  useful  because  they  specify  the  form  of 
the  optimal  control  signals.  The  fact  that  each  control  u.(t)  must  be  +1,  0,  or  - 1 
can  be  used  in  the  preliminary  design  stage  of  a  physical  system.  As  an  example, 
consider  an  attitude  control  system  for  which  gas  jets  are  to  be  used  for  control.  Then, 
it  is  not  necessary  to  build  nozzles  with  intermediate  settings  but,  rather,  nozzles  which 
open  or  close.  Thus,  the  theory  leads  to  a  design  which  is  simple  and  reliable  from  the 
mechanical  point  of  view. 

The  results  presented  up  to  now  cannot  be  used  without  additional  work  in  the 
design  of  a  practical  control  system.  The  reason  is  that  each  control  signal  u^(t)  is 
a  (nonliiioai)  fuiicLiun  of  the  signal  qj(t)  which  is  a  function  of  time  and  of  the  initial 
condition  vector  p^  of  the  adjoint  sy.stem,  .since 


g(t)  =  B'e  V  . 
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Although  the  matrices  B'  and  A'  are  known,  the  dependence  of  the  constant  vector 

and  the  terminal  state  known.  The  conceptual 

block  diagram  of  such  a  system  is  shown  in  Fig.  1.  If  for  a  given  initial  state 
and  terminal  state  the  value  of  could  be  determined,  then  the  adjoint  system 
could  be  built  and  its  output,  the  costate  £(t),  after  a  linear  transformation,  would 
generate  the  •  •  •  •‘Ij.  variables.  The  control  law  of  Eq.  1.31  implies  that 

Uj^,  u^, . .  • .  u^  are  obtained  by  passing  q^,  q^, . . . ,  q^  through  nonlinear  elements 
with  the  input-output  characteristics  of  a  relay  with  a  dead-zone. 

Additional  information  regarding  the  optimal  control  signals  can  be  obtained 
from  the  plant  described  by  Eq.  1-1.  Suppose  that  in  Eq.  1-1  the  matrix  A  has  n 
distinct,  real,  and  negative  eigenvalues.  We  denote  these  eigenvalues  by  A.^, 
i  =  1,  2, . . , ,  n.  Then,  it  is  well  known  that  each  component  p.(t)  of  the  costate  p(t) 
win  have  tlie  form 

n 

PjCt)  =  ^  . " 

k=l 

where  the  constants  a  will  be  functions  of  the  eigenvalues  A, ,  A„, . . . ,  A  and  of 

IK  1  2  n 

the  initial  conditions  p^^  ,  P2^ . p^^  .  From  Eq.  1-25  we  can  determine  that 

each  signal  qj(t)  will  be  of  the  form 

n 

=  y  ;  j  =  l,2,  ...,r  1-34 

J  ^  J«v 

k=l 

where  the  constants  p  are  functions  of  the  a.,  andb...  Since  the  A 's  were 

JK  ik  ij  k 

assumed  to  be  real,  negative,  and  distinct,  it  is  well  known  that  each  function  q.(t) 


on  the  initial  state 
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is  zero  at  most  n-1  times.  Then,  it  is  clear  that  the  control  signal  Uj(t)  will  be  zero 
at  most  n-1  times.  However,  if  the  function  qj(t)  changes  sign  exactly  m  times, 

0  <  m  sn-1,  then  the  control  u^(t)  will  be  zero  at  most  m  times. 

If  some  of  the  eigenvalues  of  A  are  complex  numbers  with  non-positive  real 
parts,  then  there  is  no  upper  bound  on  the  times  that  each  signal  qj(t)  will  change 
sign  :  hence,  there  exists  no  bound  on  the  times  that  each  control  m(t)  is  zero. 

Each  function  qj(t)  is  continuous.  From  Eq.  1-31  and  the  continuity  of  each 
qj(t),  we  conclude  that  control  sequences  of  the  form 


u.(t)  = 


0  <  t  <  t. 


tj=.'t<t2 


cannot  be  optimal,  since  there  must  exist  a  finite  interval  of  time  during  which  the 
control  Uj(t)  is  zero.  In  other  words,  if  at  present  m(t)=+l  is  the  correct  control, 
and  if  at  some  time  in  the  future  Uj(t)  =  -  1  is  required,  the  change  of  the  control 
signal  from  +1  to  - 1  cannot  be  instantaneous  but  the  control  must  be  zero  for  some 
finite  time  before  it  can  jump  to  -  1.  However,  control  sequences  of  the  form 


0  s  t  <  t. 


'2  '  '  *  ‘3 


can  be  optimal. 
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1.7  Uniqueness  of  the  Optimal  Control. 

In  this  section  we  will  discuss  the  uniqueness  of  the  optimal  control  functions. 
In  general,  there  exists  more  than  one  control  function  which  will  force  the  plant  from 
an  Initial  state  to  a  terminal  state  using  the  same  minimum  amount  of  fuel.  Each  of 
these  fuel~optimal  controls  will  require  a  certain  amount  of  time.  We  will  show  that 
the  theory  developed  will  lead  to  the  control  function  which  requires  not  only  the  least 
amount  of  fuel  but,  in  addition,  the  least  amount  of  time. 

In  this  Section  we  were  concerned  with  the  minimization  of  the  functional 


1-37 


Consider  the  minimization  of  the  functional 


1-38 


which  implies  the  minimization  of  the  sum  of  the  response  time  and  of  the  fuel 
required.  The  Hamiltonian  correspondir^  to  Eq.  1-38,  is 

r 

H'  =  1+^1  Uj(t)  1  +  <  A^f.p  >  +  <  Bu, p  >  .  1-39 

j-1 


The  minimization  of  the  Hamiltonian  H',  given  by  Eq.  1-39,  with  respect  to  the 
control  u(t),  leads  to  the  control  law  given  by  Eq.  1-31,  which  was  obtained  from  the 
minimization  of  the  Hamiltonian  given  by  Eq.  1-20.  Thus,  it  is  apparent  that  the 
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control  law  of  Eq.  1-31  will  result  in  a  control  function  which  will  force  the  plant  from 
an  initial  state  to  a  final  state  using  the  least  amount  of  fuel  and  requiring  the  least 
amount  of  time.  An  example  of  the  above  theory  is  given  in  Section  3.  5. 

1,  8  The  Time -Varying  Case. 

We  have  derived  the  optimal  control  law  for  time -invariant  plants.  In  this 
Section  we  .shall  show  that  the  results  obtained  carry  through  to  the  case  of  time- 
varying  plants . 

If  the  controlled  plant  is  time -varying  but  linear,  then  its  matrix  differential 
equation  is 

.1.  ~  '  1  "40 


The  Hamiltonian  is 


r 

H  =  ^  I  Uj(t)  I  +  <  A(l)^,  p  >  +  <  B(t)_u,p  >  . 
j=l 


The  adjoint  system  satisfies  the  differential  equation 


p(t)  =  -  A'(t)p(t)  . 


1-41 


1-42 


If  we  define 

a(t)  =  B'(t)p(t)  , 


then  the  optimal  control  law  is  the  one  given  by  Eq.  1-31.  However,  since  the  matrices 
A(t)  and  B(t)  are  functions  of  time,  the  comments  of  Section  1.  6  regarding  the  number 
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of  changes  of  the  vector  p(t)  and  £(t)  are  not  valid. 

1.  9  Summary  and  Conclusions. 

In  this  chapter  we  have  used  the  general  theory  of  optimal  control  systems  in 
order  to  determine  the  form  of  the  optimal  control  for  minimum -fuel  operation.  We 
have  found  that  each  control  signal  must  be  +1,0,  or  -  1,  and  that  the  exact  time 
response  of  each  control  signal  depends  on  the  output  of  the  adjoint  system. 
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2.  MINIMUM  FUEL  CONTROL  OF  A  SINGLE  INPUT  PLANT 
2. 1  Introduction. 

In  this  Section  we  will  examine  the  minimum  fuel  control  of  a  plant  described 
by  a  transfer  function  G(s)  containing  no  zeroes.  The  input  to  the  plant  is  the  (single) 
control  u(t).  The  similarity  transformation  will  be  used  in  order  to  determine  a 
more  convenient  set  of  state  variables. 

2.  2  Definitions  and  Terminology. 

We  wish  to  control  a  plant  described  by  the  transfer  function 


G(s) 


1 

(s-X,)(s-X  )...(s-\) 
1  z  n 


2-1 


Let  y^(t)  be  the  output  of  the  plant  and  let  u(t)  be  the  input  signal,  i.  e. ,  the  control, 
to  the  plant.  Assume  that  the  control  signal  is  bounded  by  unity. 


lu(t)  I  <  1 


2-2 


The  output  y^(t)  and  the  control  u(t)  are  related  by  the  n-th  order  differential 
equation 

(D  -  (D  -^2)  •  •  ■ 

02r 

{D*’  +  a^_^  d”  ^  +  .  . .  +  a^D  +  a^  }  y^(t)  =  u(t)  2-4 

A  d 

where  D  r  is  the  differential  operator. 
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Now  we  will  define  a  set  of  variables  such  that  the  differential  equation  2-4 
will  be  in  the  form  of  Eq.  1-1.  Let 


y.(t)  ^  D^'^j(t) 


i  =  li  2,  . . . ,  n  .  2-5 


Using  Eqs.  2-5  and  2-4,  we  may  write  the  matrix  differential  equation 


1 

0 


-a 


1 


0 

1 


-a 


2 


0 

^1 

0 

0 

^2 

+ 

0 

a  1 

n-1 

u(t) 

which  is  of  the  form 


2,  = 


2-7 


where  the  n-vector  ^  is  defined  by  Eq.  2-5  and  its  elements  are  the  output  yj^(t) 
and  n-1  of  its  time  derivatives.  The  matrix  A  is  defined  by  2-0.  The  matrix  B 
is  the  identity  matrix  and  the  column  vector  u(t)  is 
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Now  we  will  use  the  well-known  similarity  transformation  in  order  to  define 

a  new  set  of  variables.  Assume  that  the  poles  X, ,  . of  the  transfer 

12  n 

function  0(8)  are  distinct.  Note  that  the  eigenvalues  of  the  matrix  A  are  also 
^2’ '  ‘  there  exists  a  diagonal  matrix  A  of  the  form 


and  a  nonsingular  matrix  P  of  the  form  (the  Vandermonde  matrix) 
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such  that 


A  =  P'^  AP  .  2-11 

Define  a  new  set  of  variables  x, ,  x„, . . .  ,x  by 

1  z  n 

X  =  Z  2-12 

Then,  it  is  easy  to  show  that  the  x  variables  satisfy  the  matrix  diffe.tential  equation 

X  =  Ax  +  P"^u(t)  .  2-13 

Let  the  n-th  column  of  the  matrix  P  ^  be  the  column  vector  r  =  (r,,  r r  ). 

—  ~  1  2  n 

Then,  Eq.  2-13  together  with  Eq.  2-8  yield 

Xi  =  X^x^  +  r^u(t)  ;  i  =  1, 2, . . . ,  n  2-14 

It  is  easy  to  show  that  the  constant  is  the  residue  at  the  pole  A.  of  the  transfer 
function  G(s),  i.e., 


r.  = 
1 


(A.  -  A,). . .  (A.  -  A.  ,)  (A  -  A  . .  (A.  -  A  ) 
M  r  '  i  i-l' '  i  l  +  r  '  i  n' 


2-15 


Physically,  each  variable  x.  is  a  linear  combination  of  y,ty,i  •  •  •  ,y  as  indicated 

I  A  M  ri 

by  Eq.  2. 12. 

If  some  of  the  eigenvalues  (or  poles)  A^  are  the  same,  then  the  A  matrix 

Q 

is  not  diagonal,  but  it  is  of  the  Jordan-Canonical  form. 

?  B.  Friedman  "Principles  and  Techniques  of  Applied  Mathematics",  J.  Wiley  and  Sons, 
New  York,  1956,  pp  76-89. 
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In  this  section  we  have  defined,  for  future  ease  of  computation,  a  new  set  of 


variables  which  in  the  rest  of  this  Section  we  will  use  as  the  state  variables.  Note 
that  each  value  of  the  ^-variables  yields  a  unique  value  for  the  x-variables  and 
vice-versa. 

2.3  Minimum -Fuel  Control. 

The  problem  which  we  want  to  solve  is;  Given  an  initial  value  of  the  output 
and  n-I  of  its  time  derivatives,  i.e. ,  an  initial  state  and  a  desired  fixed  terminal 
state  then  determine  a  control  u(t),  1  u(t)  |  si,  which  will  force  the  plant  from 
Zq  Zf  which  will  minimize  the  functional 


F  = 


2-16 


The  initial  condition  will  define  a  initial  state  ^  using  Bq,  2-12. 

Similarly,  the  terminal  state  will  define  a  terminal  state  x^.  Hence,  the  problem 

of  minimum-fuel  control  may  be  restated  as  follows:  Find  the  control  u(t),  |u(t)  |  s  1. 

which  will  force  a  system  desc.ribed  by  Eq.  2-14  from  an  initial  state  x  to  a  fixed 

-o 

terminal  state  ^  such  that  the  performance  index  of  Eq.  2-16  is  minimum. 

For  the  sake  of  brevity,  define  4  =  (I  ^ ■  •  •  >  ^  ) 

1  2  n 


A 

-  x  . 
“  -o 


2-17 


The  HttiniUuiiiaii  fui  Luis  piublein  i» 


H  =  |u(t)  I  +  ^  +  u(t)  ^  r,p. 

J=1  j=l 


The  costate  £  is  defined  by 


=  -  A'£  =  -  A p 


hence , 


Pj(t)  =  -  A.p^(t) 


i  =  l,  2, .  . . ,  n 


For  the  sake  of  brevity,  define  ri  =  ,  t)  ). 

X  M  n 


TJ  |p(0) 


2-21a 


Then,  solution  of  2-21  yields 

-X.t 

P.(t)  =  T],e  ' 


i=  1,  2,  .  . . ,  n 


To  minimize  the  Hamiltonian  H,  we  follow  the  steps  in  Section  1.  5  and  find  the 
optimal  control  law. 


u(t)  =  0 


sgn  {1  -  !  q(t)  1  }  =+l 


u{t)  =  -  sgn{q(t)  }  if  sgn  { 1  -  !  q(t)  1  }  =  -  1 


q(t)  ^  |r,p.(t)=  I  r,,.6 
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Thus,  the  control  signal  u(t)  must  be  +1,0,  or  -1  in  order  to  accomplish  a  given 
control  action  with  a  minimum  amount  of  fuel.  The  comments  of  Section  1.  6  apply 
in  this  case  too. 

2. 4  Evaluation  of  the  Hamiltonian. 

In  this  Section  we  shall  show  that  the  Hamiltonian  defined  by  Eq.  2-18  is  a 
constant  scalar  function  if  the  control  law  Eq.  2-23  is  used. 

Consider  the  equation  of  the  x -variables 


X.  =  Xx.  +  r,u(t) 
1  11  i  ' 


Case  h  If 

u(t)  =  0 

then 

x.(t)  =  e 


V 


i  =  1,  2, . . . ,  n 


Hence,  the  Hamiltonian  H,  using  Eqs.  2-26,  2-27,  and  2-22,  is 
n  n  ,  ,  n 


i=l 


Y  -V 

?  i?.e 
i=l  i=l 


H  =  Y  Xx.p.  =  Y  \i.e  S.e  ^  =  Y 

,0111  oil  1  ^111 


which  is  a  constant. 

Case  If 

u(t)  =  A  =  +  1 

tlieii 


<,(t)  -  [s, 


+ 


1 


A.t 


for  i  =  1,  2, . . . ,  n  . 


2-25 


2-26 

2-27 


2-28 


2-30 
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Hence,  the  Hamiltonian  is 


n 

H  =  I  A  I  +  ^ 

i=l 

and  after  some  manipulation, 

n 

H  =  1  +  ^  XjIlTii  + 
i=l 

From  Eqs.  2-28  and  2-32  we  conclude  that  during  the  response  of  a  minimum 
fuel  system, the  Hamiltonian  is  piecewise  constant.  From  Theorem  5  of  Ref.  6,  which 
states  that  the  Hamiltonian  must  be  continuous  and  piecewise  differentiable,  it  follows 
that  the  Hamiltonian  is  a  constant. 

2.  5  Discussion  of  the  Results 

The  unknown  relationship  between  the  initial  conditions  t;.  of  the  adjoint 
system  and  of  the  initial  states  of  the  controlled  plant  makes  the  actual  design  of 
a  minimum-fuel  control  system  impossible  without  additional  work.  It  is  expected 
that  the  difficulties  encountered  in  the  design  of  minimum  time  (or  bang-bang)  control 
systems  will  also  be  found  in  the  design  of  minimum -fuel  systems.  The  author  has 
shown  in  Ref.  7  that  for  the  minimum  time  problem,  the  relation  between  the  state 
of  the  plant  and  that  of  the  adjoint  system  cannot  be  found  unless  the  problem  of 
control  has  been  completely  solved  using  the  concent  of  switching  sets.  In  the  case 
of  the  minimum  time  problem,  the  concept  of  the  switching  sets  has  been  used  to 
advantage . 

In  the  following  two  Sections  we  will  examine  two  second-order  systems  and 


+  A 


1  'ft 


i=l 


1  'I’i  ■ 


i=l 


2-31 


2-32 
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design  the  minimum  fuel  control  using  geometrical  concepts.  The  design  is  not 
carried  out  in  terms  of  the  adjoint  system,  but  the  results  obtained  in  Sections  1  and  2 
are  fully  exploited  in  order  to  simplify  the  arguments  which  are  based  on  the  geometrical 
point  of  view. 

In  addition  to  the  comments  and  conclusions  of  Section  1.6,  the  following 


conclusions  may  also  be  obtained. 

Suppose  that  all  the  eigenvalues  (or  poles)  X^,  . . .  ,X^  are  either  negative 
or  have  negative  real  parts.  If  the  desired  final  state  is  the  origin  of  the  j?(or  x) 
state  space,  that  is,  we  v'ish  to  bring  the  output  and  its  n-1  time  derivatives  to  zero, 
then  u(t)  =  0  is  the  optimal  control.  This  is  due  to  the  fact  that  such  a  plant  will 
always  dissipate  energy  and,  theoretically,  after  an  infinite  amount  of  time,  the  output 
and  all  of  its  derivatives  will  become  zero.  Since  u(t)  =  0  was  used,  there  is  no  fuel 


consumed.  If  we  examine  the  costate  £  in  Eq.  2-22,  we  observe  that  each  term  of 
-Xt 

the  form  e  ^  will  become  infinite  for  any  i),  Since  physical  considerations 

lead  to  the  conclusion  that  u(t)  =  0  must  be  used  in  order  to  reach  the  origin,  it 


follows  that  for  thi.s  problem  =  .  . .  =  =  0.  In  short: 


( I )  If_  Re[  X,]  <  0  ,  i  =  1,  2 . n,  and  h  Xf  "  £  ’  then  u(t)  =  0  ,  0  <  t  <  »  , 

leads  to  F  =  0,  which  implies  that  tj.  =  0,  for  all  i  =  1,  2,  .  . .  ,n,  hence  p,(t)  =  0, 
for  all  i. 


Now  suppose  that  the  terminal  point  is  not  the  origin,  but  is  a  point  of  the 
solution  of  the  homogeneous  matrix  differential  equation  x  =  ^x  or  x  =  Ax,  which 


implies  that 


2-33 
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or 


2-34 


Once  more,  physical  considerations  lead  to  the  result  that  u(t)  =  0  is  the  optimal 
control  in  the  Interval  0  s  t  s  t^  since  no  fuel  is  used.  There  is  no  restriction  on 
the  eigenvalues  Then,  since  t^  may  be  quite  large,  the  initial  conditions  of  the 
adjoint  system  must  be  zero,  i.  e.  ,  =  0,  for  all  i  =  1,  2, . . . ,  n,  which  Implies  that 

p^(t)  =  0  for  all  i=  1,  2, . . . ,  n  and  all  t  ,  0  ^  t  <  t^.  In  short; 

(II)  If  ®  1 »  ~  0  -  t  leads  to  F  =  U,  which 

implies  that  7] .  =  0,  for  i  =  1, 2 . .  and  p.(t)  =  0  ,  for  all  i  =  1,  2, . . . ,  n, 

and  all  t  in  the  interval  0  <  t  t^  . 
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3.  MINIMUM-FUEL  CONTROL  OF  A  DOUBLE  INTEGRAL  PLANT 


3.  .1.  Introduction. 


In  this  Section  we  will  examine  in  det  ail  the  optimal  minimum-fuel  control 

2 

of  a  plant  with  the  transfer  function  G(s)  =  1/s  ,  which  in  the  control  literature  is 


known  as  a  double- integral  plant.  Using  the  phase  plane  as  a  guide,  the  actual  design 


of  the  controller  will  be  obtained. 


3.  2  The  Equations  of  Motion. 


We  assume  that  the  controlled  plant  has  the  transfer  function 


G(s)  =  --  .  3.1 

s 


Let  Xj(t)  be  the  output  and  u(t)  be  the  input  (and  control)  signal.  Assume  that 


|u(t)I  <  1  .  3.2 


Define 

dx  (t) 

V*’  =  — s  --  ■ 


Since  the  plant  is  governed  by  the  differential  equation 


d"x  (t) 

— ^ -  =  u(t),  3.4 

dt 


then  the  states  x^  and  satisfy  the  matrix  differential  equation. 


“ 

0  1 

^1 

■1- 

r 

0 

Xi(O)  =  ^ 

;  3.5 

L  X  J 

0  0 

-’'2  . 

u(t) 

X2(0)  =  42 
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Equation  3.  5  is  in  the  Jordan  Canonical  form;  hence,  no  further  transformation  of 
variables  is  necessary  . 

The  fuel  consumed  is  given  by 


F  = 


The  adjoint  system  satisfies  the  differential  equation 


3.6 


Pj(0)  =  T)^ 
PjCO)  > 


3.7 


which  we  may  solve  readily  and  obtain 


Pjft)  =  hj  =  constant 
P2(t)  = 


3.8 


The  optimal  control  law  is 
u(t)  =  0 

u(t)  =  -  sgnip^fr)  } 


if 

if 


sgn{l  -  Ip^ft)  I  }  =  +1  ^ 
sgn{i-lp2(t)  1  }  =  -1 


3.9 


Hence,  for  minimum  fuel  control 


u(t)  =0  or 

u(t)  =  A  =  +1. 


3.10 
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Now  we  may  solve  Eq.  3-5. 

Case  1:  If  u(t)  =  0  ,  then 

Xj(t)  =  +  ? 

X2(t)  =  ^2  J 

Case  _2:  If  u(t)  =  A  =  +  1,  then 

Xi(t)  =  4^  +  42*^  'f"  ~2 

=  ^2 

The  time  t  may  be  eliminated  in  Eqs.  3-11  and  3-12,  and  the  response  of  the  system 
may  be  described  by  trajectories  in  the  phase  plane.  The  arrows  on  the 

trajectories  indicate  direction  of  motion  for  positive  time.  In  Fig.  2  the  trajectories, 
for  u(t)  -  0,  are  shown.  The  trajectories  are  straight  lines  for  which  x  =  constant. 
Eliminating  the  time  t  from  Eq.  3-12,  we  obtain  the  trajectory  equations 

^  4  ■  T  ^ 

The  above  trajectories  are  parabolas  in  the  -  x^  plane.  They  are  illustrated  in 
Fig.  3.  The  solid  curves  are  for  A  =  +  l,  while  the  broken  curves  are  for  A  =  -  1. 

3.3  Information  From  the  Adjoint  System. 

From  Eq.  3-9  it  is  quite  evident  that  the  control  depends  on  the  function 
P2(t).  From  Eq.  3-8 


3-14 


and  since  tj^  and  17^ 


are  (unknown)  constants,  it  is  clear  that  Eq.  3-14  is  the  equation 
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of  a  straight  line.  In  Fig.  4  six  different  "shapes"  of  the  function  P2(t)  are  shown 
for  different  ranges  of  the  constants  r]^  and  Using  Fig.  4  and  Eq.  3-9,  we 
conclude  that  the  following  control  functions  are  optimal. 


u(t) 


=  i 


0 

-1 


0  <  t  <  t, 


t  ^  t  s  t 
1  -  ^ 


3.15 


u(t)  = 


0 

+1 


0  <  t  <  t. 


t  S  t  £  t 
I  f 


3-16 


u(t)  = 


0  <  t  <  t 


1 


t^  s  t  <  t^ 


V 


3-17 


1  ■' 

o 

lA 

rt 

A 

u(t)  = 

t^^t<t2  1 

^  ,  3-18 

1  +1 

1 

u(t)  = 

0 

0  <  t  s  tj 

3 -18a 

u(t)  - 

-1 

0  £  t  <  tj 

3-19 

u(t)  = 

+  1 

0  £  t  £  tj  . 

3-20 

We  have  found  that  examination  of  the  costate  as  a  function  of  time  has 
resulted  in  six  possible  forms  of  the  optimal  control  function.  The  times  t^,t2,  and 
tj  in  the  above  equations  are  usually  not  the  same.  We  shall  call  control  signals  of 
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the  same  type  as  those  given  by  Eqs.  3-15  through  3-20  allowable. 

3. 4  The  Origin  as  the  Terminal  Point. 

In  this  Section  we  consider  the  problem  of  reaching  the  origin  of  the  -  x^ 
phase  plane  from  any  initial  condition  with  a  minimum  amount  of  fuel. 

Consider  all  points  in  the  phase  plane  -  x^  with  the  property  that  if 
u(t)  =  0  is  applied,  then  the  resulting  trajectory  will  pass  through  the  origin.  We  let 
X  =  X  =  0  in  Eq.  3-11  and  we  obtain  the  equations 

X 


hence, 


«2  =“ 


3-21 


3-22 


We  conclude  that  the  only  point  from  which  one  may  reach  the  origin  using  u(t)  =  0 
is  the  origin  itself. 

Consider  all  points  in  the  phase  plane  from  which  one  may  reach  the  origin 
using  u(t)  =  +  1.  Substituting  A  =  +  1,  =  X2  =  0  in  Eq.  3-13,  we  obtain 


^1  2  ^2 


3-23 


Since  the  motion  must  be  for  positive  time,  Eq.  3-23  is  valid  only  in  the  region 


i  >  0  ;  <  0  3-24 

I  i! 

The  curve  defined  by  Eq.  3-23  and  3-24  is  shown  in  Fig.  5  and  it  is  denoted  as  the 
curve. 
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Consider  all  points  in  the  phase  plane  from  which  one  may  reach  the  origin 
using  u(t)  =  -  1.  Substituting  A  =  -  1,  =  0  in  Eq.  3-13,  we  obtain 

I  V 

=  3-25 

Since  the  motion  must  be  for  positive  time  only,  Eq.  3-25  is  valid  only  in  the  region 

£  0  ;  ^2  “ 

The  curve  defined  by  Eqs.  3-25  and  3-26  is  referred  to  at  the  v  curve  and  is 
shown  in  Fig.  5. 

Now  we  will  establish  the  optimality  of  the  curves  Y_^  and  y  using  the 
results  of  Section  3.  3;  that  is,  we  shall  show  that,  if  (I,,  e  v  ,  then  u(t)  =  +  1 
is  the  optimal  control  policy  and  if  (§,, !_)  e  y  ,  then  u(t)  =  -  1  is  the  optimal 
policy.  Suppose  that  (I, ,  4.,)  is  on  the  y,  curve;  then  the  allowable  controls  are  given 

X  ^  •f'  ” 

by  Eqs.  3-15  through  3-20.  It  is  easy  to  see,  using  Fig.  2  and  Fig.  3,  that  any  control 
of  the  form  of  Eqs.  3-15  through  3-19  will  not  bring  the  state  (4^^,  4^)  to  the  origin. 
Hence,  u(t)  =  +  1  is  optimal,  and  the  optimality  of  has  been  established.  One 
may  use  the  same  argument  and  show  that  if  (4  j,  ^2}  ^  V_  >  then  u(t)  =  -  1  is  the 
optimal  control  policy;  hence  y_  is  an  optimal  minimum  fuel  trajectory.  To 
recapitulate: 

If  (4,1  4n)  ^  y.  >  then  the  conUol  u(t)  -  -r  1  will  force  the  state  to  {0,  0)  v.dth 
X  A  *T  — — -  — — — —  — -  ~  ■  — — —  —  ■  ■ 

a  minimum  amount  of  fuel.  If  e  y  ,  then  the  control  u(t)  =  -  1  will 

force  the  state  to  (0, 0)  with  a  minimum  amount  of  fuel. 


Consider  Region  IV  in  Fig.  S  defined  by 


2  ^2 


>0 


X2  <  0 


Suppose  that  the  initial  state  (|  4^)  is  in  Region  IV.  Consider  the  allowable  controls 

given  by  Eqs.  3-15  through  3-20.  It  is  easy  to  conclude  that  controls  of  the  type  given 
by  Eqs.  3-15,  3-17,  3-19,  and  3-20,  cannot  transfer  the  state  point  (^2^.12^ 
origin.  Thus,  we  must  consider  the  ailowable  controls  defined  by  Eqs.  3  -16  and  3-18. 
The  control  function  described  by  Eq.  3-16  implies  that  u(t)  =  0  must  be  applied  until 
the  state  point  is  on  the  curve  and  u(t)  =  +  1  from  the  point  of  intersection  until 
the  origin  is  reached.  This  control  policy  results  in  the  trajectory  ABO  in  Fig.  6. 
The  control  function  given  by  Eq.  3-18  will  result  in  a  trajectory  of  the  form  of  ACDO 
in  Fig.  6.  Now  we  shall  show  that  for  any  in  Region  IV,  the  optimal  policy 

is  to  use  u(t)  =  0  until  the  state  point  is  forced  on  the  y  curve  and  then  to  apply 
u(t)  =  1. 

The  fuel  consumed  in  order  to  bring  the  point  A  -  (4^,  in  Fig.  6,  to  the 

point  11  on  y,  is 

+• 

’’ah  ■  “  3-2* 

since  u(t)  -  0  was  used.  The  time  required  to  take  point  B  to  the  origin  O  using 
u(t)  •-  1  is,  from  Eq.  3-12, 


’^AB  “  ”  ^2  -  '  ^2'  • 
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Hence,  the  fuel  to  take  point  B  to  the  origin  is 


CAD 

^  UU<-AB  =  >«2l 


Hence,  the  total  fuel  consumed  in  order  to  bring  point  A  =  (4  origin  is 


^ABO  ^  ^AB  ^BO  “  '^2' 


Now  consider  the  trajectory  ACDO  in  Fig.  6.  Let  point  C  have  the  coordinates 
C  =  (S',  4i).  Then  point  D  has  the  coordinates  D  =  (Sj'  .  Clearly 

X  A 

42  <  ^2  ’ 


From  Eq.  3-12 


£  -  4' 
^2  ^2 


Therefore, 


F^C  =  1^2'-'^2' 


F  =0 
CD 


3-3Sa 


^DO  ”  ^2' 
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Hence,  the  total  fuel  along  the  trajectory  ACDO  is 


ACDO 


^2' 


llj  +1^;1  =2 


^^1-1 


3.37 


and  it  is  clear,  because  of  Eq.  3-32,  that 


^ABO  ^ACDO  . 


3.38 


The  conclusion  is: 

If.  ^2)  hi  Region  IV,  defined  by  Eq.  3-27,  then,  for  minimum  fuel  control 
apply  u(t)  =0  until  the  curve  reached  and  then  apply  u(t)  =  +  1. 

Consider  Region  II  in  Fig.  5.  Region  II  is  defined  by 


2~  ^^2 


Xi  <  0  ;  X2  >  0 


3.39 


Using  identical  arguments  as  above,  one  may  draw  the  conclusion 

Region  II,  defined  by  Eq.  3-39,  then,  for  minimum  fuel  control , 
apply  u(t)  =  0  until  the  curve  ^  reached  and  then  apply  u(t)  =  -  1. 

The  trajectory  EGO  in  Fig.  6  is  an  example  of  a  minimum-fuel  trajectory  . 

We  consider  points  in  Region  I,  defined  by 


X 


1 


>  - 


X 


2 


0 


I 
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Examine  Eqs.  3-15  through  3-20.  From  Figs.  2  and  3  it  is  apparent  that  controls  of 
the  type  given  by  Eqs.  3-15,  3-16,  3-17,  3-19,  and  3-20,  cannot  force  the  state  point 
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to  the  origin.  The  only  type  of  control  left  is  that  of  Eq.  3-18  which  is 


u(t)  = 


0  £  t  <  tj 
tj  s  t  <  t 


2 

f 
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The  time  tj  at  which  the  control  changes  from  -  1  to  0  must  be  such  that  the  state 
point  is  in  Region  IV.  Otherwise,  the  state  point  cannot  be  forced  to  the  origin. 
Furthermore,  for  minimum  fuel  control  it  is  easy  to  see  that  the  control  must  change 
from  -  1  to  0  at  a  point  ,  -6),  where  6  is  a  very  small  positive  number.  In 
Fig.  6  the  trajectory  HJIO  is  optimal.  The  control  u(t)  =  - 1  forces  the  state  from 
H  =  ^2)  to  J  =  6).  At  J  the  control  u(t)  =  0  is  applied,  v/hich  forces  the 

state  from  J  =  (I ',  -6)  to  I  =  ,  -6).  The  point  I  is  on  the  curve  y^.  Hence, 

application  of  u(t)  =  + 1  will  force  the  state  from  I  to  the  origin.  It  is  easy  to  verify 
that  the  fuel  consumed  is 


''hIIO  =  l«2l  +  “  ” 

since  6  is  a  very  small  positive  number.  Hence,  we  arrive  at  the  following 
conclusion: 

If_  (I  Ij)  is  in  Region  I,  defined  ^Eq.  3-40,  then  the  optimal  control 
policy  is  to  apply  u(t)  =  -  1  until  the  state  point  just  enters  Region  IV,  defined  by 
Eq.  3-  27,  then  apply  u(t)  =  0  which  will  force  the  slate  to  the  curve  y_^  ;  at_  the 
Instant  the  state  is  on  y^  ,  apply  u(t)  =  +  1  in  order  to  force  it  to  the  origin. 
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Finally,  we  consider  initial  conditions  in  Region  III,  defined  by 

3-43 

Using  similar  arguments  as  above,  we  draw  the  conclusion. 

If  is  in  Region  III,  defined  by  Eq.  3-43,  apply  u(t)=+l  until  the 

state  is  forced  into  Region  II;  apply  u(t)  =0  until  the  state  is  forced  onto 
the  y  curve,  and  apply  u(t)  -  -  1  in  order  to  bring  the  state  to  the  origin. 
The  trajectory  KLMO,  in  Fig.  6,  illustrates  this  policy. 

The  above  results  are  summarized  in  Table  1 . 


TABLE  I 


Initial  Conditions  in 

Optimal  Control  for  Minimum  Fuel. 

Region  I 

(1)  Apply  u(t)  =  ”  1  to  just  enter  Region  IV 

(2)  Apply  u(t)  =  0  to  go  to  y  curve 

(3)  Apply  u(t)  =  + 1  to  go  to  (0  , 0)  . 

Region  11 

(1)  Apply  u(t)  =  0  to  go  to  y _  curve 

(2)  Apply  u(t)  =  -1  to  go  to  (0, 0). 

Region  HI 

(1)  Apply  u(t)  =  +  1  to  just  enter  Region  II 

(2)  Apply  u(t)  =  0  to  go  to  y_  curve 

(3)  Apply  u(t)  =  -1  to  go  to  (0*  0)  . 

Region  IV 

(1)  Apply  u(t)  =0  to  go  to  y  curve 

(2)  Apply  u(t)  =+l  to  go  to  "^(0,0)  . 

Apply  u(t)  =  +  1  to  go  to  (0, 0). 

y. 

A.pply  u(t)  =  -1  to  go  to  (0, 0). 

-llO- 


3.3  The  Non-Uniqueness  of  the  Optimal  Control  Function. 

In  the  previous  Section  we  have  arrived  at  a  method  of  control  based  on  the 
values  of  the  state  variables .  The  control  functions  thus  defined  will  bring  the  state  to 
the  origin  using  a  minimum  amount  of  fuel. 

Now  we  will  pose  the  question:  Do  there  exist  other  control  functions  which 
will  bring  the  state  point  to  the  origin  using  the  same  minimum  fuel  but  different 
response  times?  The  answer  to  this  question  is  YES.  To  illustrate  the  point,  we 
consider  Fig.  7.  Suppose  that  our  initial  point  is  at  A  =  (1,  -1).  Consider  the 
admissible  controls  which  lead  to  the  trajectories  ABO,  ADEO,  and  AFGO.  The 
trajectory  ABO  is  obtained  through  the  use  of  the  theory  of  the  previous  section,  i.  e. , 
u(t)  =  0  in  the  segment  AB  and  u(t)  =+l  in  the  segment  BO.  The  trajectory  ADEO 
is  obtained  by  using  u(t)  -  +  1  in  the  segment  AD,  u(t)  =  0  in  the  segment  DE,  and 
u(t)  =  +  1  in  the  segment  EO.  This  control  sequence  +1,0,  +  1  is  a  violation  of  the 
control  suggested  from  the  adjoint  system.  Similarly,  the  trajectory  AFGO  is  obtained 
by  using  u(t)  =  +  1  in  the  segment  AF,  u(t)  -  0  in  the  segment  FG,  and  u(t)  =  -1  in 
the  segment  GO.  Table  2  below  contains  the  response  time  and  the  consumed  fuel 
along  each  trajectory. 


TABLE  II  (Refer  to  Fig.  7) 


Trajectory 

Consumed  Fuel 

I'ime 

A.BO 

1.000 

1.500 

ADEO 

1,  000 

1,625 

AFGO 

1.000 

2.250 

From  the  above  table  it  is  evident  that  all  three  trajectories  require  the 
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same  amount  of  fuel  but  different  response  time.  The  trajectory  ABO  is  the  one 
which  requires  the  least  amount  of  time.  This  phenomenon  is  to  be  expected  as 
pointed  out  in  Section  1.7.  Hence,  the  control  law  of  Table  1  will  result  in  a 
minimum-fuel  trajectory  which  takes  the  least  time  to  reach  the  origin. 

3. 6  The  Design  of  the  Controller 

In  this  section  we  shall  present  the  elements  of  design  of  a  controller  which 
will  force  any  initial  state  to  the  origin  with  a  minimum  amount  of  fuel.  Tlie  design 
is  carried  out  using  the  results  of  Section  3. 4. 

A  possible  physical  realization  of  the  controller  is  shown  in  Fig.  8.  In  this 
figure  the  output  of  the  plant  G(s)  is  differentiated  in  order  to  obtain  x^  =  Xy 
The  signal  x  passes  through  a  square-law  nonlinearity,  which  may  be  realized  by 

A 

a  biased  diode  network,  and  then  through  an  attenuator  to  produce  the  signal 
— X2  1  .  The  signal  and  x^l  x^l  are  added  to  produce  the  signal  z. 

If  the  signal  z  is  positive,  then  the  state  point  is  in  Region  1  of  IV;  if  the  signal  z 
is  negative,  then  the  state  point  is  in  Region  II  or  III  (refer  to  Fig.  5).  We  pass  the 
signals  z  and  x^  through  identical  relays  and  add  their  outputs  and  then  change  the 
sign  to  obtain  the  signal  w.  The  signal  w  is 

f  -2  if  {xyx^  is  in  Region  I 

w  =  0  if  is  in  Region  II  or  IV  3-44 

y^+2  if  iG  in  Region  III 

The  signal  w  is  the  input  to  a  relay  with  a  dead  zone, as  shovaiin  Fig.  8,  whose  output 


drives  the  plant. 


This  suggested  design  requires  a  differentiating  network,  a  square -law -type 
nonlinearity  (which  may  be  a  biased  diode  network)  two  summing  amplifiers,  one  inverter, 
two  ideal  relays,  and  one  relay  with  dead  zone.  It  is  slightly  more  complicated  than  the 
controller  required  for  the  bang-bang  control  of  t  he  same  pi  ant . 

3.  7  Arbitrary  Terminal  States 

In  the  previous  Sections  we  have  examined  the  minimum -fuel  control  of  the 
2 

plant  G(s)  =  1/s  when  the  terminal  point  was  at  the  origin  of  the  phase  plane.  In 
this  Section  we  shall  examine  the  control  policy  when  the  terminal  point  is  not  the 
origin.  Note  that  due  to  the  double  integrating  action  of  the  plant  G(s),  the  plant  is 
completely  controllable;  i.e. ,  given  any  initial  condition  and  any  terminal  state,  one 
may  reach  that  terminal  state  with  controls  ]  u(t)  I  <  1 . 

Suppose  that  the  terminal  point  is  T  =  (1, 1),  as  shown  in  Fig.  9.  We  define 
three  curves  and  as  follows: 

7^  is  the  locus  of  all  points  in  the  plane  from  which 
one  may  reach  T  using  u(t)  =  0. 

7_^  is  the  locus  of  all  points  in  the  plane  from  which 
one  may  reach  T  using  u(t)  =  +1. 

7_  is  the  locus  of  all  points  in  the  plane  from  which 
one  may  reach  T  using  u(t)  =  -1. 

The  curves  y^,  y^,  and  y_  are  shown  in  Fig,  9.  These  curves  also  define 
Regions  I,  II,  III,  and  XV,  as  shov/n  in  Fig,  9.  It  is  easy  to  show  that  for  minimum- 
fuel  control  and  the  least  time  of  response,  the  control  policy  given  in  Table  3  is  the 
optimal  one. 


TABLE  III  (Refer  to  Fig.  9) 


Optimal  Control  Policy 
Apply  u(t)  =  0  to  reach 
Apply  u(t>  =  +  1  to  reach 
Apply  u(t)  =  -1  to  reach 


T. 

T. 

T. 


(1)  Apply  u(t)  =  -  1  to  just  reach  Region  IV. 


Region  I  (2)  Apply  u(t)  =  0  to  reach  y  curve. 

(3)  Apply  u(t)  =  -I-  1  to  reach 


Region  II 

1 - 

(1)  Apply  u(t)  =  0  to  reach  y  curve  . 

(2)  Apply  u(t)  =  -  1  to  reach  T. 

Region  III 

(1)  Apply  u(t)  =  +  1  to  reach  y  curve. 

(2)  Apply  tt(t)  =  0  to  reach  T.  ° 

Region  IV 

(1)  Apply  u(t)  =  0  to  reach  y  curve. 

(2)  Apply  u(t)  =  +  1  to  reach 

A  controller,  of  analog  nature,  is  shown  in  Fig.  9A.  The  controller  was 
designed  using  the  information  of  Table  3  and  of  Fig.  9.  The  signal  passes  through 
a  nonlinearity  which  duplicates  the  and  y_  of  Fig.  9.  The  output  signal  of  the 
nonlinearity  is  compared  to  the  measured  value  of  to  yield  the  signal  z.  If  z  is 

positive,  then  the  state  point  is  to  the  right  of  the  y_  and  y^  curves.  If  z  is 
negative,  the  state  point  is  to  the  left  of  the  y_  and  curves.  The  signal  v 
indicates  the  polarity  of  the  signal.  The  x^  signal  is  compared  with  x^^  and 
w  is  the  sign  of  the  difference.  But, 


(a) 

If 

z  >  0 

and 

V  -  +1, 

the  state  is  in  Region  I. 

(b) 

If 

z  >  0 

and 

v=  -1. 

the  state  is  in  Region  IV. 

(c) 

If 

z  <  0 

and 

w  =  +  1, 

the  state  is  in  Region  II. 

(d) 

If 

z  <  0  and 

II 

1 

the  state  is  in  Region  III. 

The  remaining  relays,  multipliers,  and  summers  make  a  logical  network  which 
guarantees  that  the  control  law  of  Table  3  is  used. 

3.8  Summary  and  Conclusions . 

In  this  Section  we  have  examined  the  optimal  control  of  a  plant  with  the 
2 

transfer  function  G(s)  =  1/s  .  We  have  used  the  available  information  furnished  by 
the  adjoint  system  in  order  to  establish  an  optimal  minimum-fuel  control  policy  for 
any  set  of  initial  condltlon.s.  The  method  of  analy.si.s  was  primarily  geometrical  in 
nature  and  it  was  necessary  to  use  the  shape  of  the  trajectories  in  the  phase  plane 
resulting  from  the  allowable  values  of  the  control  function  in  order  to  design  the 
controller,  which  can  be  of  analog  or  digital  nature.  Thus,  any  set  of  initial  conditions 
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can  be  forced  to  zero  using  a  minimum  amount  of  fuel.  Furthermore,  among  the 
control  policies  leading  to  minimum  fuel  response,  the  controller  automatically 
selects  the  one  which  require  the  least  amount  of  response  time. 

The  physical  computer  suggested  is  only  slightly  more  complicated  than  the 
one  for  minimum  time  control  of  the  same  plant. 


4.  MINIMUM-FUEL  CONTROL  OF  THE  PLANT  G(s)  =  - - -  . 

(s  +  l)(sH-2) 

4. 1  Introduction. 

In  this  Section  we  examine  the  minimum-fuel  control  of  a  second-order 
plant  described  by  the  transfer  function 


G(s) 


1  ? 
(s  +  l)(s  +  2) 
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This  plant  is  not  conservative,  as  the  one  described  in  Section  3,  but  it  continually 
dissipates  energy.  The  optimal  control  of  this  plant  will  be  derived  again  through  the 
use  of  the  phase  plane. 

4.  2  The  Equations  of  Motion. 

We  are  given  the  plant  described  by  the  transfer  function  of  Eq.  4-1.  Let 
yj(t)  be  the  output  of  the  plant  and  u(t)  be  the  input  (control  signal)  to  the  plant.  We 
a.ssume  that 

I  u(t)  I  £  1  .  4-2 


The  output  y^(t)  is  the  solution  of  the  differential  equation 

(D  +  l)(D  +  2)yj(t)  =  p^+3D  +  2)yj(t)  =  u(t)  4-3 

where  D  =d/dt  is  the  differential  operator.  We  define  the  variables 

V  fri  =  V 

"I''  ■'1'' 

4-4 

y2(t)  =  yi(0 

$  The  numbers  1  and  2  in  the  transfer  function  were  chosen  for  computational  facility. 

Many  algebraic  equations  turn  out  to  be  quadratic,  and  roots  may  be  determined  easily 
without  the  use  of  digital  computers.  Although  the  ideas  presented  will  be  true  for 
arbitrary  values  for  the  poles,  the  actual  computation  of  various  curves  will  be  more  difficult. 
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which  satisfy  the  matrix  differential  equation 


4-5 


which  is  of  the  form  ^  =  A +  Bu  . 

Let  A  be  the  diagonal  matrix 


A  r: 


0 

-2 
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Then  there  exists  a  nonsingular  matrix  P 


with  the  inverse  P 


such  that 


4-7 


4-8 


A  =  p'^  AP 


4-9 


Define  the  state  variables  x,,x  by 


„-l 

1  =  I  1 


4-10 
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which  implies 


Xj=  2yi  +  y2) 


*2  '■''r>'2 


4-11 


Then,  the  x-variables  satisfy  the  differential  equation 


or 


=  •XjH-u(t) 
=  ~2x2-u(t) 


x^(0)  n 
XjP)  I 


4-12 
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It  is  desired  to  determine  the  optimal  control  u(t)  which  will  force  the 
system  described  by  Eq.  4-12  from  an  initial  state  (4^^,  ^2)  to  a  terminal  state 
T  =  (x  X2P  such  that  the  fuel 


•0 


is  minimum. 

The  Hamiltonian  H  is  given  by 


H  =  Iu(t)  I  +  <  A  X,  p  > 


+ 


-kg- 


>  4-15 


where  the  costate  £  is  the  solution  of  the  adjoint  system 


p  =  -  A  £  =  -  A  £ 


£(0)  =  '£  4-16 


which  implies 


Pj(t)  = 


P2(0  =  ^2^ 


2t 
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The  optimal  control  law  is 


u(t)  =  0 


if 


u(t)  = -sgn  ■[  Tj^e*^ -Tj^e  I  if 


1  t  2t  I  ^ , 

I  ^ 


I.  4-18 


Therefore,  for  minimum-fuel  control 


u(t)  =  0  or 
u(t)  =  A  =  -  1 


4-19 


Substituting  Eq.  4-19  in  Eq.  4-13,  we  find: 
Case  1:  If  u(t)  =  0,  then 


x^(t)  =4^6 


X2(t)  = 


•2t 


4-20 
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Case  2; 


If  u(t)  =  A  =  -  1,  then 


Xj^(t)  =  “  A)e  *  +  A 

XjW  =  («2  +  0-5A)e'^‘ -0.5A 


4-21 


Eliminating  the  time  t  from  Eq.  4-20,  we  obtain  the  equations  of  the  free 
trajectories 


4-22 


These  trajectories  are  shown  in  Fig.  10.  The  arrows  indicate  motion  of  the  state 
points  for  positive  time. 

Eliminating  the  time  t  from  Eq.  4-21,  we  obtain  the  equations  of  the  forced 
trajectories 

X2  --=“0.5A+  (42  +  0'  5A) 


-  A 


4-23 


The  forced  trajectories  are  shown  in  Fig-  11  for  A  =  +  l  and  in  Fig.  12  for  A  =  -  1. 

4.  3  Information  From  the  Adjoint  System. 

The  control  law  of  Eq.  4-18  indicates  that  the  value  of  the  control  depends  on 
the  function 

A  L 
q(t)  -  n^e 

Examination  of  the  possible  forms  of  q(t),  for 


nf 


4-24 


all  values  of  tj  ,t]  and  time,  leads  to 
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the  following  allowable  forms  of  the  control  u(t)  : 


u(t)  =  0 


0  <  t  S  t. 


u(t)  =  +  1 


0  :<  t  <  t. 


u(t)  =  -  1 


0  <  t  <  t. 


u(t)  = 


0  <  t  <  t. 


t  •=  t  <  t 

1“  “  f 


0  <  t  <  t, 


0  £  t  <  t. 


u(t)  = 


0  <  t  <  t, 


r  <  t  <  t 
"  f 


)  ■' 
u(t)  =  { 

I  0 


0  s  t  <  I, 


t  <  t  <  t 

1  “  f 


t  +1 

u(t)=  f  0 


0  rS  t  <  t, 


t^sr<t2 
^2^  t  s  t^ 


u(t)=  /  0 


0<t<  t^ 
t^^t<t2 

•^2="  '  ^  'f 
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u(t)  = 


u(t)  = 


U(t)  = 


u(t)  = 


j 


+1 

0 

+1 


0  £  t  <  t. 


t  < 

t  ^ 


4-34 


-1 

0 

-1 


0  £  t  <  t. 


tjS  t  <  tj 


4-35 


0  £  t  <■  t. 


t^£  t  < 
'2-  '  "  '3 


4-36 


-1 


t  *=  t  *=  t 

3“  “  f 


0  £  t  <  t 


1 


t^£  t  < 
^2“  ^3 


4-37 


t  *<  t  •*=^  t 
3~  ~  f 
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4.  4  The  Origin  as  the  Terminal  Point. 

The  problem  for  which  the  origin  is  the  terminal  point  is  trivial  since  u(t)  =  0 
must  be  applied,  as  explained  in  Section  2.  5. 

4.  5  Terminal  Point  T  =  (0. 4,  -0.2). 

In  this  section  we  consider  the  problem  of  forcing  the  plant  from  an  arbitrary 
initial  state  terminal  state  T  =  (0. 4,  -  0.  2).  We  will  analyze  the  system 

for  that  specific  terminal  point  in  order  to  present  in  the  clearest  way  the  concepts 
which  will  be  required.  Otherwise,  the  symbols  and  terminology  might  overshadow 
the  essential  concepts  and  ideas  and  lead  to  confusion. 

Define  three  curves  called  the  y  ,y  ,  and  y  curves  as  follows: 

o  + 

y  is  the  locus  of  all  points  in  the  x,  -x  phase  plane  which,  using 
O  X  M 

u(t)  =  0,  are  forced  to  the  terminal  point  T  -(0.4,  -  0.  2).  The 
equation  of  the  curve  is 

r  2 

"2  =  -°-n-0-r]  =-l'25x^  4-38 

and  is  defined  in  the  region 

>  0.4  ;  x^  s  -0.2  .  4-39 

y^  is  the  locus  of  all  points  in  the  x^^  -  plane  which,  using 
u(t)  =  -  1,  are  forced  to  the  terminal  point  T  =  (0.  4,  -0.2). 

The  equation  of  the  y_  curve  is,  from  Eq.  (4-23), 

X  .  =  0.5 -0.  8.34  (x.  -  1)^  4-40 

^  X 

in  the  region  defined  by 

Xj  <  0.4  ;  >  -0.2  4-41 
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7  is  the  locus  of  all  points  in  the  -  x^  plane  which,  using 
u(t)  =  -  1,  are  forced  to  the  terminal  point  T  =  (0. 4,  -  0.  2). 

The  equation  of  the  y  curve  is,  from  Eq.  4-23, 

x^  =  0.  5  -  0.  357(x^  +  1  4-42 

in  the  region  defined  by 

9 

Xj  >  0. 4  ;  x^  £  -  0. 2  4.43 

The  curves  y  ,y  ,  and  y  are  shown  in  Fig.  13. 

It  is  clear  that  the  curve  y  is  optimal.  Hence,  if  (4,,|„)e  y_,  then 
i!(t)  =  0  is  the  minimum  fuel  optimal  control  policy. 

Now  we  will  determine  the  fuel  required  to  force  the  plant  from  an  initial  state 
at  the  origin,  i.  e. ,  at  (0, 0),  to  the  point  T  =  (0.  4r0, 2).  Tlie  control  policy  of  Eq.  4-30 
must  be  followed.  Hence,  if  the  initial  state  is  at  the  origin,  then  we  must  apply 
u(t)  =4-  1  until  the  curve  is  reached  and  then  switch  to  u(t)  =  0.  It  is  easy  to 
compute  that  the  fuel  required  is 

F  =0.84  units^^  4-53 

m 

But  we  know  that  any  initial  state  in  the  x^-x^  plane  may  be  taken  to  the  origin  with 
zero  fuel.  Hence,  we  may  draw  the  following  conclusion: 

In  order  to  reach  the  terminal  state  T  =  (0. 4,  -  0. 2)  from  an^ 
initial  state,  tnc  maximum  fuel  required  is  0.  84  units. 


9  Through  error  in  numbering,  Eq.4-43  is  followed  by  Eq.  4-53. 
99  All  numerical  results  are  within  slide  rule  accuracy. 
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Consider  all  Initial  states  (^  ,  I- )  which  may  be  forced  to  the  y  curve 

\  0 

using  u(t)  =  +i.  Since  along  the  crajeclories  the  liiue  elapsed  is  equal  to  the  fuel 
consumed,  the  equation  of  the  above  trajectories  is,  from  Eq.  4-21, 


Xj=  l  +  (ij-l)e“^ 

-O.S  +  ci^  +  O.Sje 


4-54 


But  the  curve  has  the  equation 

X-  =-1.25x^  .  4*54a 

2  1 

Therefore,  for  intersection  of  the  trajectories  defined  by  Eq.  4.7  54a  with  the 
curve 

^  2 

-0.5  +  (l2  +  0-5)e’^^  =  -  1.25[  1 l)e’^]  4-55 

or 

^2 

^2  = -0.5  +  0. -  1.25  +  -  1  ]  .  4-56 

Equation  4-56  defines  a  family  of  curves  in  the  phase  plane  with  F  as  a  parameter. 

We  shall  call  these  curves  equal -fuel  curves,  since  any  initial  state  satisfying 

Rn.  4-.56  may  be  taken  to  T.  using  the  control  policy  of  Eq.  4-30,  with  the  same 

amount  of  fuel  F.  In  Fig.  14  we  show  some  equal-fuel  lines  for  the  values  F  =  0. 

F  =  0.  2,  F  =  0.  5,  F  =  0.  84,  and  F  =  l.O.  Note  that  the  F  =  0  curve  is  the  y  curve 

o 

and  that  all  equal  fuel  curves  terminate  on  the  y^  curve.  The  F  =  0.  84  curve  passes 
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through  the  origin. 


Now  consider  ajrjr  free  trajectory 


X 


2 


4-57 


Each  of  the  free  trajectories  will  Intersect  many  equal  fuel  lines.  Denote  the  points 
of  intersection  by  (x^,  X2  ).  Then, 

2 

^  X j  =  -  0.5  +  0.5e^^  -  1.25  [e^+x^  -  1  ]  4-58 


Equation  4-58  is  quadratic  in  Xj^  :  hence, 


X,  =  ^  [b  ±  -j  -  4ac  ] 


where 


a  =  )3  +  1.  25 
b  =  2.50  (1  -  e^) 
c  =0.75e  -  2.50e  +  1.75 


If 


b 


2 


4ac  =  0, 


4-59 


4-60 


4-61 


4-62 


4-63 


then  the  free  trajectory  will  be  tangent  to  an  equal -fuel  line.  Equation  4-63  implies 

(2.50  -  3)3)  e^-'  +  10)3e^  -  2.  50  -  7;8  =  0  4-64 


which  is  quadratic  in 


F 


with  the  two  solutions 
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and 


1 


4-65 


F 

e 


F  _  2.  50  +  7/3 

®  ■  2. 50  -  3/3 


The  point  of  tangency  (x^.x^)  is  given  by 


2.  50(1  -  e^) 
2(|3  +  1.25) 


4-66 


4-67 


F  F 

where  e  is  given  by  Eq.  4-66.  The  solution  e  =  1  in  Eq.  4-65  implies  F  =  0, 

and  it  represents  the  tangency  point  at  the  origin  between  any  free  trajectory  and  the 

extension  of  the  y  curve;  hence  it  is  discarded, 
o 

Substituting  Eq.  4-66  into  4-67,  we  find 


X,  = 


=  /3x 


2.  50  -  3/3 
2 


4-68 


We  plot  the  curve  defined  by  Eq.  4-68  in  Fig.  15,  and  we  denote  it  as  the  y^  curve. 
It  is  plotted  for  P  in  the  range 


_  ^  ^  o  r* 

-  oo<  p  -l.ZO 


A  _An 

*»  \j  y 


To  recapitulate: 

(1)  We  have  found  the  locus  of  all  points  {i,  which  may  be  forced 
to  the  curve  with  u(t)  =  +  l  and  which  require  the  same  fuel  F 
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order  to  reach  the  terminal  point  T  =  (0.  4,  -  0.  2).  Equation  4-56  is  the 
equation  of  these  curves.. 

(2)  We  have  determined  the  equal -fuel  curve  which  is  tangent  to  a  given 
free  trajectory.'  This  tangent  equal -fuel  line  is  described  by  F  obtained 
from  Eq.  4-66,  and  the  point  of  tangency  (Xj.x^)  is  given  by  Eq.  (4-68).  The 
locus  of  all  (x^,X2)  points  is  the  curve  in  Fig.  15. 

The  next  step  in  the  analysis  is  to  find  whether  the  curve  y^  is  optimal.  We 
will  have  to  use  all  the  facts  assembled  up  to  now  in  order  to  be  able  to  make  this 
decision. 

Suppose  that  the  initial  state  (|  ,|_)  is  on  the  y  curve,  described  by 
Eqs.  4-40  and  4-41.  We  know  that  the  control  policy  u(t)  =  +1  will  force  the  state 
1^,12)  to  T.  In  Fig.  14  the  point  Q  =  (-0.  4, 1.  15)  is  the  intersection  of  the 
F  =  0. 84  equal -fuel  curve  and  the  y_^  curve.  From  the  point  Q  there  are  two  ways 
of  reaching  T  using  the  same  amount  of  fuel  equal  to  0.  84  units.  One  way  is  to 
apply  u(t)  =  +1  until  T  is  reached.  Tlie  other  way  is  to  apply  u(t)  =  0  until  wc  reach 
the  origin,  then  apply  u(t)  =  +1  until  we  reach  the  curve  y^,  and  then  apply  u(t)  =  0 
to  reach  T.  The  former  policy  requires  the  least  time. 

Now  suppose  that  (^ , ,  |„)  t  y  in  the  region  defined  by 

-0.4  <  x^  <  +  0.4  .  4-70 

At  (|j,42)  terminates  an  equal -fuel  line.  Let  the  fuel  required  be  denoted  by  F.  If 
the  control  u(t)  =  0,  instead  of  u(t)  =  +1,  is  applied  at  (|^,  ^2)  then  the  resulting  free 
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trajectory  will  intersect  equal-fuel  lines  with  parameter  F  such  that 

A 

F  >  F  4-71 

as  is  apparent  from  Figs.  10  and  14.  The  same  thing  happens  if  u(t)  =  -1  is  applied. 
Thus,  we  may  draw  the  conclusion: 

If  e  in  the  region 

Xj  <  -0.4  .  4-72 

if  u(t)  a  +1  is  applied,  then  the  fuel  required  will  be  greater  than  0.  84  units.  Hence. 
u(t)  =  +1  is  not  the  optimal  control,  since  we  can  always  reach  T  with  a  maximum 
amount  of  fuel  equal  to  0.  84  units. 

Consider  all  points  in  the  plane  .such  that  they  may  be  taken  to  the  curve  y^ 
using  u(t)  =  -1.  We  shall  denote  these  points  by  (|j,  ^2)-  Their  equation  is 


^  ?  17 

X2  =  (l2  -  0-5)6  +0.5 


Since  the  y^  curve  has  the  equation 


X2  =  -1.  25  Xj  , 


it  follows  that 


-2P 

/F  nc\-^^iOC_  locl/f  1 

-r  0*  o  —  ”  i.  Us,  -f  vC  1 


f  I 


or 


^2  =  0.  5  -  0.  5e^^  -  1.25  [^^  +  1  -  J 


4-73 


4-74 


4-76 
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Equation  4-76  is  the  equation  of  another  family  of  equal -fuel  curves.  They  are 
plotted  in  Pig.  16.  They  terminate  on  the  y_  curve.  The  reason  that  the  line 
F  0.262  is  included  is  as  follows:  Consider  the  intersection  of  the  curve  with 
the  axis.  The  point  of  intersection  is  at  (0.  22, 0).  The  fuel  required  to  go  along 
the  Y^  curve  to  the  terminal  state  T  is  0. 262  units . 

Now  consider  points  to  the  right  of  the  Y  and  y_  curves  and  in  the  fourth 
quadrant  of  the  plane.  It  is  easy  to  see  that  the  optimal  control  policy  is  to 

apply  u(t)  =  0  until  the  Y^  curve  is  reached  and  then  to  apply  u(t)=+l.  The 
intersection  of  the  free  trajectory  with  the  curve  will  occur  in  the  region 


0.22  <  x^  <  0.4  ■ 

■ 

0  <  x^  <  -0.  2  . 


4-77 


and  the  maximum  fuel  required  is 

F*  =  0.262  units.  4-78 

m 

Now  consider  points  in  the  region  between  the  Y^  and  Y  curves .  There  are  three 
ways  of  forcing  them  to  T. 

(1)  Apply  u(t)  =  -1  until  y^  is  reached  and  then  apply  u(t)  -  0. 

(2)  Apply  u(t)  =  0  until  y  is  reached  and  then  apply  u(t)  =  -1. 

(3)  Apply  u(t)  =  0  until  y^  is  reached  and  then  apply  u(t)  =  1. 

Now  we  shall  show  that  policy  (2)  above  is  not  optimal  with  respect  to  (1),  Suppose  that 

(Ij^,  1^)  is  on  the  equal-fuel  curve  with  F  =  F^^.  Equation  4-76  is  the  equation  of  a 

F  2F 

parabola  with  vertex  at  the  point  (e  -1, 0.  5-0.  5e  ),  which  is  a  point  in  the  fourth 
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quadrant  of  the  x  -x  plane.  It  follows  that  a  free  trajectory  will  intersect  the  y_ 

X  M 

curve  at  a  point  for  which  F  =  F2  such  that 


F  >  F 

1 


Now  we  shall  compare  policies  (1)  and  (3)  above.  Given  a  point  (1^^,  ? 2)  in  the  region 

between  y  and  y  curves,  If  u(t)  =  -1  is  applied  until  the  y  curve  is  reached,  then 
0  -  0 

the  fuel  required  is  found  to  be,  from  Eq.  4-76 


,50(12  +  1)  -  V  1-71  -  2.50^^  -5 


Now  suppose  that  at  ( 4 , ,  4  ,)  we  apply  u(t)  =  0  until  the  y  curve  is  reached,  in  the 
region  defined  by  Eq.  4-77,  and  then  u(t)  =  +1  is  applied  to  reach  T.  Let  the  fuel 
required  be  denoted  by  F ' .  The  free  trajectory  has  the  equation 


The  y^  curve  is  defined  by 


Xj^  =  i  -  0.  6e 


X2  =  -  0.  5  +  0. 3  e 


Hence, 


-  0.5  +0.3e 


F' 

1  -  0.  6e 
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F' 

which  may  be  solved  for  e  to  yield 


F' 

e 


4-84 


The  locus  of  all  points  such  that 

F  F' 
e  =  e 


4-85 


have  the  property  that  either  policy  (1)  or  policy  (3)  will  require  the  same  amount  of 
fuel.  This  locus  is  denoted  as  the  curve  and  is  shown  in  Fig.  17.  Note  that 
terminates  at  the  terminal  point  T  =  (0. 4,  -0.2).  We  may  draw  the  following 
conclusions: 

If  the  initial  state  is  between  the  y^  and  y^  curves,  then  the  optimal 
control  policy  is  to  apply  u(t)  =  -1  until  the  curve  y^  is  reached  and 
then  to  apply  u(t)  =  0. 

If  the  initial  state  is  to  the  right  of  the  y^  curve,  the  optimal  control 
policy  ^  to  apply  u(t)  =  0  until  the  curve  y^  is  reached  and  then  to 
apply  u(t)  =  +1 . 

Since  the  y_  curve  is  to  the  right  of  the  y^  curve,  it  follows  that  the 
curve  y  ^not  optimal . 

In  Fig.  18  we  show  the  division  of  the  phase  plane  into  regions  of  operation  for 
optimal  control.  As  a  partial  summary  we  identify  the  various  curves  shown  and  their 
equations . 
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V^;  The  defining  equation  of  the  curve  is 


=  - 0.5  +  0.  834(xj-l) 


in  the  region 


-0.4  <  Xj  <  +  0.  4 


-0. 2  <  <  +  1. 15 


y^  :  The  defining  equation  of  y^  is 


X2=-1.25x^ 


in  the  region 


>  0.4 


X2<-0.2 


4-86 


4-87 


4-88 

4-89 


y 


l' 


The  curve  is  a  portion  of  the  F  =  O.  84  equal-fuel  curve.  Its 
equation  is,  letting  F  =  O.  84  in  Eq.  4-56 

_2 

X2  =  2.3 -1.25[  1.365  +Xj  I  4-90 


in  the  region 

-0.  4  <  Xj  <  O 
0<  X  <  1.15 


4-91 


y^  '.  The  curve  y^  in  the  locus  of  the  t.angency  points  of  free  trajectories 
with  equal-fuel  curves.  Its  equation  is,  from  Eq.  4-68, 


x^  =  -y  j^2.  50  Xj  -  5  Xj^  4-92 

in  the  region 

0<x,<0.8  7  ^.,3 

-0.8<x,<0  J 

2 
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y^:  The  curve  y^  is  obtained  from  Eqs.  4-80,  4-84,  and  4-85,  and  is 
valid  in  the  region 

0. 4  <  X-  ;  X  >  -  0. 2  .  4-94 

X  IM 

In  Fig.  18  we  also  define  three  regions  of  operation-  Region  I  is  the  slice  between  the 

y  and  V.  curves,  including  the  curve  but  not  the  y  curve.  Region  II  is  the  set 
0  o  a  0 

of  points  enclosed  by  the  y,,  y,,y„,  and  y  curves.  The  curves  y  and  y  are 
parts  of  Region  II,  but  not  the  curves  y^  and  y^.  Region  III  is  the  rest  of  the  space. 

We  shall  now  present  the  optimal  control  law  using  the  region  in  Fig.  18.  The 
optimal  control  is  given  in  Table  4. 


TABLE  IV  (Refer  to  Fig.  18) 


Initial  State  in 

Control  Policy 

Apply  u(t)  =  +  1  to  reach  T. 

y 

0 

Apply  u(t)  =  0  to  reach  T. 

Region  I 

1)  Apply  u(t)  =  -1  to  reach  y  curve. 

2)  Apply  u(t)  =0  to  reach  T.° 

Region  11 

1)  Apply  u(t)  =  +1  to  reach  y  curve. 

2)  Apply  u(t)  =0  to  reach  T. 

Region  III 

— 

1)  Apply  u(t)  =  0  to  reach  Region  11  or  y^  curve. 

2)  Apply  u(t)  =  +1  to  reach  y  or  T. 

3)  If  y^  is  reached,  apply  u(t)  =  0  to  reach  T. 

We  have  already  discussed  how  the  optimal  policy  in  Region  I  was  derived.  Let 
us  now  explain  the  control  policy  in  Region  11. 
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Consider  any  state  (?  ^2)  Region  II.  From  Fig.  14  it  is  evident  that  the 
point  (4ji  I2)  equal  “fuel  curve  with  0  :<  F  «0.  84.  Initial  application 

of  u(t)  =  0  or  u(t)  =  -1  will  result  in  a  trajectory  which  will  intersect  equal-fuel  curves 

A 

with  F  >  F.  A  little  thought  will  convince  the  reader  that  u(t)  =  +1  is  the  optimum 
control . 

Now  we  will  examine  states  in  Region  III.  Consider  the  points  in  the  first 
quadrant  and  in  Region  III.  The  application  of  u(t)  =  0  will  force  them  to  the 
curve.  Any  other  policy  will  require  more  fuel.  Consider  the  points  in  the  second 
quadrant  and  in  Region  III.  If  the  initial  state  is  to  the  right  of  a  free  trajectory 
terminating  at  the  point  Q,  then  u(t)  =  0  will  force  the  state  point  to  the  y^  curve. 
Otherwise,  u(t)  =  0  will  force  them  to  the  origin.  From  the  origin  u(t)  =  +1  will 
force  the  state  to  etc.  Points  in  the  third  quadrant  will  be  forced  to  the  origin  by 
u(t)  =  0.  In  the  fourth  quadrant  initial  states  to  the  right  of  the  y  and  y,  curves 
will  be  forced  to  y^  with  u(t)  =  0,  as  already  explained.  Now  consider  initial  states 
in  the  fourth  quadrant  and  Region  III.  At  the  curve  y^  the  free  trajectory  is  tangent 
to  an  equal-fuel  curve.  From  the  shape  of  the  equal-fuel  curves  in  Fig.  14,  it  is 
evident  that  the  control  u(t)  must  change  from  0  to  +1  on  the  curve  y^. 

4, 6  Summary  and  Conclusions. 

We  have  discussed  in  detail  the  optimum  control  of  a  plant  with  the  transfer 
function  of  Eq.  4-1.  The  major  part  of  the  Section  4  was  devoted  to  the  study  of  the 
minimum-fuel  control  for  a  terminal  point  not  at  the  origin  of  the  phase  plane.  It  was 
necessary  to  introduce  the  concept  of  the  equal-fuel  curves  in  order  to  arrive  at  the  optimal 
control  policy.  It  was  found  that  the  phase  plane  was  divided  into  three  regions  of 
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operation,  and  that  for  each  region,  an  optimal  control  was  specified  in  Table  4. 
The  whole  problem  was  examined  with  the  greatest  detail  possible  in  order  to  bring 
into  focus  the  steps  which  must  be  followed  in  problems  similar  to  the  one  treated. 
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5.  CONCLUSIONS  AND  SUGGESTIONS  FOR  FUTURE  RES  EARCH 


In  this  report  we  have  shown  that  for  minimum-fuel  control  of  a  linear  time 
invariant  plant,  each  component  of  the  control  vector  should  have  the  values 
+1,  0,  or  -1  if  unity  is  the  bound  on  the  magnitude  of  the  control  vector  components. 
Furthermore,  we  have  shown  that  the  magnitude  and  polarity  of  the  control  depends  on 
the  output  of  the  adjoint  system. 

We  have  used  the  properties  of  the  output  of  the  adjoint  system  in  order  to 
analyze  the  optimal  response  of  two  second-order  plants.  The  analysis  was  primarily 
geometrical  and  was  carried  out  in  the  phase  plane,  using  as  a  guide  the  sysleiii 
trajectories  and  the  concept  of  the  equal -fuel  curves. 

One  of  the  second-order  systems  examined  was  a  double  integral  plant.  It  was 
found  that  the  analysis  in  the  phase  plane  was  quite  easy  and  a  complete  controller  was 
designed  using  the  results.  The  other  system  examined  was  a  second-order  plant  with 
two  real  and  negative  poles.  It  was  found  that  the  phase  plane  can  be  separated  into 
three  regions  of  operation  and  that  in  each  region  corresponds  an  optimal  value  for  the 
control  function. 

5.2  The  Unsolved  Problems. 

The  first  problem  which  must  be  solved,  in  the  opinion  of  the  author,  is  the 
general  second -order  system.  The  second -order  system  with  arbitrary  distinct  real 
poles  will  give  a  lot  of  insight  into  the  operation  of  high-order  systems.  Second-order 
systems  with  complex  poles  are  very  important  and  they  should  also  be  examined  in 
detail.  The  experience  and  results  obtained  from  the  two  examples  in  this  report 
indicate  that  the  concepts  of  equal -fuel  lines  and  of  the  region  of  operation  will  be 
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helpful  in  the  determination  of  the  optimal  feedback  control  function. 

A  natural  follow-up  is  the  design  of  higher-order  systems.  However,  if 
high-order  systems  are  expensive,  and  if  the  use  of  digital  computers  is  economically 
justifiable,  then  a  set  of  control  equations  could  be  obtained  which  may  be  solved  by 
the  digital  computers. 

In  this  report  we  have  assumed  that  the  response  time  was  not  fixed.  There 
are  many  problems  for  which  the  response  time  is  fixed,  and  one  is  interested  in 
determining  the  control  which  will  accomplish  the  required  control  task  in  the  given 
fixed  time  with  a  minimum  amount  of  fuel.  For  such  problems  the  results  given  in 
this  report  can  be  used  to  advantage,  but  their  extension  is  not  trivial.  The 
requirement  that  the  response  time  is  fixed  would  require  different  regions  of 
operation,  which  will  be  defined  by  the  equations  of  both  equal -fuel  curves  and  equal  - 
time  curves  (isochrones).  The  control  functions  for  these  fixed-time,  minimum-fuel 
systems  will  again  have  the  values  +1,  0,  or  -1.  Similar  problems  to  the  above  are 
systems  for  which  the  fuel  is  fixed  and  it  is  required  to  minimize  the  response  time. 
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FIG.  1  CONCEPTUAL  BLOCK  DIAGRAM  OF  A  MINIMUM 
FUEL  SYSTEM  USING  THE  ADJOINT  SYSTEM. 


FIG.  5  PLOT  OF  THE  y.  and  y  CURVES  DEFINING  THE  FOUR 

.J. 

REGIONS  OF  OPERATION  IN  THE  PHASE  PLANE. 


3-2 


FIG.  6  EXAMPLES  OF  MINIMUM  FUEL  TRAJECTORIES 
(EXCEPT  FOR  TRAJECTORY  ACDO). 


FIG.  7  EXAMPLE  OF  EQUAL  FUEL  TRAJECTORIES,  WITH 
DIFFERENT  RESPONSE  TIMES.  (FUEL  F  ■  1.000) . 


8  THE  FEEDBACK  CONTROL  SYSTEM  FOR  MINIMUM 
CONTROL  OF  THE  PLANT  1/s^.  THE  TERMINAL 
STATE  IS  X  ■  0,x„  -  0. 


■IG.  9a.  THE  FEEDBACK  CONTROL  SYSTEM  FOR 
MINIMUM  FUEL  CONTROL  OF  l/s^  TO  AN 
ARBITRARY  TERMINAL  STATE. 


FIG.  11  PHASE  PLANE  TRAJECTORIES  OF  l/(s+l)(s+2) 
FOR  u(t)-+l. 
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